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ABSTRACT 

Because of the continually changing environment of a space station, visual feedback is a vital element of a 
telerobotic system A real-time visual servoing system would allow a telerobot to track and manipulate randomly 
moving objects. This paper develops methodologies for the automatic selection of image features to be used to visu- 
ally control the relative position between an eye-in-hand telerobot and a known object. A weighted criteria function 
with both image recognition and control components is used to select the combination of image features which pro- 
vides the best control. Simulation and experimental results of a PUMA robot arm visually tracking a randomly mov- 
ing carburetor gasket with a visual update time of 70 milliseconds are discussed. 


I. Introduction 

Most would agree that the eventual goal of a telerobot is to perform dangerous tasks in space which would oth- 
erwise require human intervention. To perform these tasks, telerobots must be equipped with many of the sensory 
capabilities of humans. Because of the continually changing environment of a space station, vision is undoubtably a 
very important sense. Until recently, the primary uses of vision in telerobotics have been for recognizing, locating, 
and inspecting stationary parts. Image processing equipment is now reaching the stage where vision may be used as a 
feedback signal to control the position and orientation (pose) of the telcrobot’s end-effector in real time [1][2], This 
visual feedback would allow a telerobot to manipulate and track a randomly moving part without any previous 
knowledge of the part’s placement or motion. 

The type of feedback for visual servoing systems has taken two forms [3]: position-based and feature-based. 
The traditional method has been to extract image features t, recognize the desired object by matching image features 
to a set of pre-taught features, interpret the pose of the object based on the image features, and use the error between 
desired and estimated pose to drive the system. A second method is to use actual image features instead of the part s 
interpreted position as the feedback signal for controlling the manipulator [4], With the proper selection, these 
features can be directly related to the control parameters of the robotic system. The savings in time needed to inter- 
pret the workpiece’s pose from the image features is made possible by determining the desired image features during 
an off-line CAD simulation. 

In this paper, a resolved motion rate control scheme [5] with feature-based feedback are used to visually servo a 
robot manipulator with an eye-in-hand camera over a moving object (see Figure 1). This method of visual feedback 
introduces two fundamental questions. How many image features are necessary to control the desired degrees of free- 
dom of the manipulator end-effector? And which image features would provide the best control? This paper 
addresses these two questions and develops methodologies for the automatic selection of image features used to visu- 
ally control the relative pose between the manipulator end-effector and a workpiece. The selection of these features 
depends on a blend of image recognition and control criteria. The image recognition criteria include feature robust- 
ness, completeness, uniqueness, and cost of feature extraction. The control criteria include system observability, con- 
trollability, and sensitivity. A weighted criteria function is used to select the combination of image features which 
provides the best control. Both computer simulations and laboratory experiments on a PUMA robot arm were con- 
ducted to verify the performance of the feature selection criteria. 


This work was supported in part by an IBM research fellowship and in part by the National Science Foundation under Grant CDR 8803017 to the 
Engineering Research Center for Intelligent Manufacturing Systems. Any opinions, findings, and conclusions or recommendations expressed in this 
article are those of the authors and do not necessarily reflect the views of the funding agencies. 

fin this paper, image features refer to geometric shapes in the camera’s image. Examples include circles, edges, comers, and curves. 
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II. Differential Relationship between Part’s Pose and Image Feature Points 

In the resolved motion rate control structure in Figure 1, the changes in image features are transformed into 
changes in joint angles. This transformation may be decomposed into a series of three transformations: feature to 
camera coordinates, camera to end-effector coordinates, and end-effector to robot joint coordinates. The last two 
transformations are well known [6] but the first is not. For our purposes, we will assume that a unique transformation 
from camera space to robot joint space exists. Therefore, our ability to control the pose of the robot depends on the 
differential transformation from image feature space to camera space. This section concentrates on the differential 
transformation from image feature points ft to camera coordinates. 

When analyzing the transformations from the camera space to the image feature space, consider the coordinate 
frames shown in Figure 2. The following nomenclature is used: 

(jc , y , z) = position of the part with respect to the camera frame; 

(d> , 9 , V) = roll, pitch, and yaw orientation of the part with respect to the camera frame; 

( Px i > p yi . P Pt) = position of feature point i on the part with respect to the part frame; 

( ‘ x i . c yi . Cz i) - position of feature point i on the part with respect to the camera frame; 

( >x i » ‘yd = corresponding position of the point in the image plane; 

/= focal length of the camera lens; 

lx = x axis scaling factor in pixels/mm due to camera sampling; 

1 y =y axis scaling factor in pixels/mm due to camera sampling; and 
(x„ , y a ) = the image plane offset in pixels due to camera sampling. 

We assume that the camera characteristics ( /, y x , y y , x 0 , y a ) are known. To be able to interpret the 3-D pose of a part, 
we have assumed that the spatial relationships between feature points, i.e., ( p x „• , p y, , p z,), are known from a CAD 
model. 

Geometric optics are used to model the mapping between the Cartesian space and the image feature space. The 
mapping consists of two stages: a thin lens model of the perspective transformation, and a mapping into a two- 
dimensional plane caused by camera sampling. The transformation from the camera frame, c, to the image plane, /, 
can be written as 
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where we assume that the blurring effects, the quantization, and the lens distortion effects are negligible. 

Using the differential transform technique [6], the change in image positions (d l x h d'y L ) in terms of the change 
in the part’s pose (d x , d y ,d„ 5,, ^,8,) is [7] " 
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tt The (x , y) position of a feature such as a circle or comer in the image will be referred to as a feature point 
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This expression is suitable for simulation purposes where we want to determine the change in image features given a 
small change in the part’s pose. However, for control purposes we seek an inverse solution. We would like to know 
the change in the part’s pose given the change in several feature points in the image. An exact inverse may exist if 
three feature points are considered. The linear differential relationship between the change in image feature points 
and the change in the part’s pose would then be 6 x 6 matrix J. 

Notice that this Jacobian matrix depends on the positions of the feature points with respect to the camera. For 
the inverse Jacobian matrix J to exist, all three points can not have the same x and y position in the image plane. If 
the points are collinear, then a rotation about the line would not be observable. Another restriction is that not all 
feature points can be on a plane perpendicular to the focal axis with one of the points located at the focal center. 
Again, there exists a rotation in this plane which the camera will not be able to sense. 

One final problem still remains. How can we use this transformation if we do not know the actual positions of 
the feature points with respect to the camera? In our experiments, two approaches were used. The first was to esti- 
mate their positions from their image positions and the known spatial relationships between image points. The second 
was to use the desired positions as determined by the CAD simulation. While the latter is self-explanatory, the first 
method deserves further development. 

The objective is to find the pose of the part with respect to the camera, i.e., (x, y, z, $, 0, y), from the a priori 
information, ( p x l , p y i , p z i ), and the measured feature points in the image, ( / x, , I y i ), for < = 1 , 2 , • • • ,n, where n is 
the number of feature points in the image. Variations of this "location determination problem" have appeared in 
several papers [8]. Overdetermined systems of equations are typically used to determine the "best" solution. Least 
squares techniques, the Random Sample Consensus paradigm [8], and a generalized Hough transform approach [9] 
have been proposed to eliminate errors caused by noisy images and modeling errors. 

Unfortunately, many of these methods are too time consuming for real-time use. Instead, we propose using the 
following gradient search to continually update the object’s pose. This search minimizes the sum of squared distance 
errors between four or more actual feature points and the geometrically modeled feature points in the image. Because 
of the nature of this search, a fairly good initial estimate of the part’s pose is necessary. It should also be noted that 
the resulting least squares solution is optimal if the image noise has a Gaussian distribution. However, if there are 
outliers in the data, the Random Sample Consensus approach [8] might provide more accurate results. 

The objective of this gradient search is to vary the pose of the workpiece until the image positions of the 
modeled feature points align with the actual image feature points. In mathematical terms, we would like to minimize 


F = ZFt = \ £ ('x, - u,) 2 + ('y, -v,) 2 


(4) 


where (m* , v^) is the actual image position of feature point /. The modeled position of feature point i in the image is 
determined from the camera’s perspective and sampling Eq. (2), and from the roll, pitch, yaw representation of the 
part with respect to the camera [6], 
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where c<{> s cos (<|)), 5<t> s sin (<f>), cQ = cos (0), 50 s 5in(0), cy = cos (\|/), and s \ |/ = sin (y). 

In these equations, the known variables are the camera parameters, (/\ y x » Y, , x a , y a )\ the actual image feature 
positions, (u, , v,), for i = 1 , 2 , • • • , n; and the positions of the features with respect to the part’s frame, 
( p x . f Py. # p z .) t for i = 1 , 2 , * * • , n, where n is the number of features. The unknown variables (the ones we want to 
solve for) are the pose of the part, (x ,y , z , , 0,y). For ease of notation, let x = [x ,y , z , <t>, 0 , y] r , 
y, = [ c Xi . c y, , ^,f, and z, = [ 'x t , / y,] 7 ', where the superscript T denotes vector/matrix transpose. Then Eq. (2) can 
be represented by z, = G(y,), and Eq. (5) can be represented by y, = H,(x). 
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Newton’s gradient search is used to minimize this error function with respect to the x parameters. The kth itera- 
tion of the search is given by 

\ k = X*.! - (x t ) VF t (x*) (6) 

where VF is the gradient of F with respect to x and the F„ is the Hessian matrix of F with respect to x. 

Because the relationship between the error function and the part’s pose consists of a series of composite map- 
pings described in Eqs. (2), (4), and (5), the gradient of F is the product of the Jacobian matrices of individual map- 
pings. The gradient of F with respect to x is 
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where F iZi is the Jacobian matrix of F, with respect to z,-, G,, is the Jacobian matrix of G with respect to y,, and H, x is 
the Jacobian matrix of H, with respect to x. The elements of these Jacobian matrices are given in [7], 

The chain rule for the Hessian matrix F*, is slightly more difficult. 
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The elements of these Hessian matrices are also in [7]. 


III. Feature Selection Criteria 

Since the Jacobian obtained in the previous section depends on the features’ positions in the 3-D space with 
respect to the camera, some features will provide better visual control than others as the part moves with respect to the 
camera. In addition to these control issues, image recognition plays an important role in choosing features which are 
reliable and robust. This section lists several image recognition and control criteria which should be considered in the 
selection process. This is not an all inclusive list and additional criteria could be added at a latter date. 

The image recognition criteria used in our feature selection process include [10]: 

1 . rare features (similar to unique features in [ 1 1 ]), 

2. feature set robustness (similar to likelihood of being seen in [11]), 

3. computational inexpensive features, and 

4. feature set completeness. 

To quantitatively evaluate these criteria, a set of measures was designed for the visual servoing experiments. For uni- 
formity, these measures were designed to range between zero and one with zero being the most desirable and one 
being the least desirable. 

A unique feature has an easily identifiable characteristic which differs from other features in the image. Such a 
feature is often used to quickly identify an object. This makes the feature very useful for control since the feature can 
be quickly re-identified if it is momentarily lost. A measure of feature uniqueness for a set of features [f { , • •• J m ) 
may be written as 


X\ifl » ' ' ' >/m) = 


1 

Nm(n- 1) 


m N 
i=i /= t 


(ii) 


where m is the number of features to be used for control, n is the total number of features in the images, N is the 
number of possible views during the control process, and My is the number of similar features to feature i in image j 
(0£My <, n- 1). Note that the measure X\ reaches a maximum of one if My = n - 1 for all i and j and a minimum of 
zero if My = 0 for all i and j. 
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In older that the recognition process be resilient to noise, we would like the features to be robust. For feature 
extraction systems where the image is scanned for features, feature robustness depends on the size and type of feature 
and the method of feature extraction. Usually the larger the feature is in the image, the more robust the feature is. A 
general measure of feature robustness may be written as: 
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where 0 < a„ < 1 is proportional to the size of the feature i in the image j, and 0 < p,, < 1 is related to the reliability of 
feature extraction method ij. In our experiments, features were restricted to circles in the image. Since the same 
method of feature extraction was used throughout the experiments, the reliability factor is - 1 for all i,j. Our 
measure of feature robustness for circles was 
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where fy was the radius of circle j in image j, and r m> , was the maximum radius possible. 

The computational expense of features refers to the time and space complexities of the feahire extraction pro- 
cess For most cases, space complexity is negligible. Time complexity, on the other hand, is very important for deter- 
mining the tracking ability of the visual servoing control. The shorter the time of feature extraction is, the larger the 
bandwidth of the control. For feature extraction systems where the image is scanned for features, computational 
expense also depends on the size and type of feature and the method of feature extraction. In contrast to the robust- 
ness criteria, usually the smaller the feature is in the image, the smaller the computational expense is. A general 
measure of computational expense of features may be written as: 
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where 0 < iy £ 1 is related to the time complexity of the feature extraction method ij for a feature of fixed size. In our 
experiments, the location of a circle was verified with four scan lines spanning out from the approximate origin of the 
circle to the circle’s edge. Again, since the same method of feature extraction was used throughout the experiments, 
Xij = 1 for all ij. Our measure of computational expense for the circles was 
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Notice the trade-off between the computational expense and robustness criteria. Smaller circles have less computa- 
tional cost with poor feature robustness, while larger circles have better robustness at the expense of computational 


cost. 


If a workpiece can be identified from any view point, the set of features is said to be complete. For our cir- 
cumstances where we are continually controlling the position of the workpiece with respect to the camera, only a sub- 
set of all possible views is necessary. Our measure for feature set completeness is 
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The control criteria used in our feature selection process include: 


1. Observability of the workpiece’s pose through image feature points. 

2. Controllability of the workpiece’s pose using the inverse Jacobian matrix in section II. 

3. Sensitivity of the control to noise. 

In the presense of image noise, the gradient search in section n was used to minimize the error in the image 
positions of four or more features. Since this error is measurable, it provides a means of evaluating the observability 
of the part’s pose with respect to the camera. Using the error function F in Eq. (22), our measure for the observability 
of the part’s pose is 
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and F mx is the largest acceptable error. 
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If the sampling time between image acquisitions is short and the distance between the actual and desired image 
features is small, the differential relationship in section II could be used to control the relative pose of the workpiece. 
The desired change in camera position would be determined by multiplying the difference between actual and desired 
features by the inverse Jacobian. In this respect, we will say that the pose of the workpiece is “controllable” if the 
inverse Jacobian exists and is non-singular. 

In addition to checking that a unique solution exists, we would like the equations to be well-conditioned. This 
means that for "small" changes in J and the part’s pose, the changes in image feature positions should also be small. 
This could be thought of a measure of the control’s sensitivity to noise. The condition of the Jacobian matrix J is 


c ( J) = 1 1 J| | || J-'ll (16) 

where the norm may be 1 1 j | j , 1 1 ( | 2 • or I H I ~ • Moderately small values of c (J) imply that the equations are well- 
conditioned. However, large values of c (J) do not necessarily mean that the equations are ill-conditioned. Instead, it 
just means that the equations will be ill-conditioned for some changes in the part’s pose. In general, we would like to 
choose image feature points which would minimize the condition of J. Since the condition of J will become 
extremely large as the Jacobian approaches a singular point, the condition may also be used to evaluate the controlla- 
bility of the workpiece’s pose using the inverse Jacobian matrix. Therefore, a measure for evaluating both the con- 
trollability and sensitivity of the system may be written as 
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and c m«i is the largest acceptable value for the condition of J. 


Another consideration for real-time control is the effect that changes in the feature positions have on the ele- 
ments of the Jacobian matrix. To reduce computational costs, it is desirable for the elements of the Jacobian to be 
constant or slowly time-varying. If the elements do not change substantially between two camera acquisitions, the 
inverse Jacobian does not have to be updated each sample time. Therefore, we would like to choose image feature 
points such that the elements of the Jacobian change very little throughout the motion. In other words, we would like 
to minimize the sensitivity of J to the change of image feature points, 
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where 3 bn are the elements of the Jacobian. Our measure of sensitivity to change was 
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and J max is the largest acceptable value of j(J). 


Because of the various conflicting interests of the above measures, a weighted criteria function was used to 
select the features which would provide the best control 
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where £ w, = 1 are the weighting factors which are the choice of the designer. Similar to the individual measures, 
1=1 

the best selection will be the set of features with the smallest overall measure %. 


IV. Simulation and Experimental Results 

Both computer simulations and laboratory experiments were performed to illustrate the performance of the pro- 
posed feature selection criteria. Computer simulations were used to test the performance of the feature selection cri- 
teria under ideal conditions and with additive image noise. The experiments verified the simulation results and 
showed the usefulness of the feature selection criteria for visually controlling a robot manipulator in real time. 

In our experiments, a single Pulnix TM-540 CCD camera mounted on the end-effector of a PUMA 600 robot 
was used to visually servo the robot’s end-effector over a moving carburetor gasket (the workpiece) as shown in Fig- 
ure 3. Because of their ease of feature extraction, the circles in the gasket were used as the control features. The 
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visual servo control was initiated when the camera was approximately over the gasket in a pre-taught position. After 
the gasket was recognized, the locations of three circles in the image were continually updated. The changes in cam- 
era pose needed to control the robot were determined by multiplying the difference between the desired and actual 
image positions by the inverse Jacobian in section n. The changes in camera pose were then converted to the changes 
in joint angles using the manipulator’s inverse Jacobian [12]. 

The simulations were modeled after the experiments and their objective was to determine which three image 
feature points (circles) of the carburetor gasket out of a total of seven circles (see Figure 3) would be best suited for 
controlling the pose of the gasket with respect to the camera. Table 1 lists the positions with respect to the gasket s 
frame and the radii of the seven largest holes. The number of feature points was limited to seven to reduce the 
number of possible combinations. The desired position and orientation of the gasket with respect to the camera was 
(x,y,z) = (-48,63,200) millimeters and (roll, pitch, yaw) = (-90,0,0) degrees, respectively. The camera parameters 

are given in Table 2. 

First, we evaluated the condition and sensitivity of J for all possible combinations of three circles. We found 
that the set of features which minimize c(J) form an equilateral triangle about the origin of the gasket. In particu ar, 
the set of circles {0,3,5} minimized c(J) while the set of circles {0,1,2} maximized c(J). We also found that the con- 
dition of J decreases as the gasket is moved closer to the camera. On the other hand, the sensitivity of J to change 
was minimized as the feature points moved closer to the focal axis and the x and y image axes. According to this sen- 
sitivity criteria, the best set of features would be the set of circles { 1,2,4} . In contrast to the condition of J, the sensi- 
tivity of J to change is minimized if the gasket is as far away from the camera as possible. If computation time was a 
factor, a weighted sum of the measures %6 and could be used to determine a set of features which would provide 
accurate control without updating the inverse Jacobian. 

Next, we considered the tracking response of the camera for the two sets of features, {0,3,5} and {0,1,2}. For 
both sets of features, circle 6 was used as a fourth point for determining location. To test the control process, a ramp 
input in position and orientation was applied to the gasket’s pose. The change in the x, y, and z directions was 5 mil- 
limeters per sample. The change in the roll, pitch, and yaw of the gasket was 0.9 degrees per sample. If our system 
was operating at video rate (30 Hz - both fields), this would correspond to a positional velocity of 15 centimeters per 
second and a rotational velocity of 27 degrees per second. Several cases were run with and without image noise. 

First consider the ideal case with no image noise. Table 3 shows the root mean square (RMS) error between the 
desired and actual position and orientation of the workpiece with respect to the camera. Three schemes were used to 
update the Jacobian matrix. In column 1, the Jacobian was updated with the actual positions of the feature points. In 
column 2, the Jacobian was updated only once with the desired positions of the feature points. In column 3, the Jaco- 
bian was updated with the estimated positions of the feature points using the methods in section II. The following 
conclusions were made from the ideal case: 

1 . Up datin g the Jacobian with the actual positions provides the best control. 

2. When the actual or desired positions of the features were used, the set of features {0,3,5} performed as 
well as the feature set {0,1,2}. 

3. As predicted by the large condition of the Jacobian for feature set {0,1,2), it was difficult to estimate the 
part’s pose from the feature set {0,1,2}. Therefore, for estimation purposes it is best to choose a set of 
feature points with a small condition number, such as the feature set {03,5} . 

Next consider the same simulation except add Gaussian image noise with a standard deviation of 0.5 pixels. 
The RMS error for this case is shown in Table 4. In addition to the same columns as in Table 3, a fourth column 
shows the results when seven points are used to estimate the pose of the workpiece. The columns with 00 indicate that 
the woikpiece was lost before the simulation was completed. The following conclusions can be made from the noisy 
image case: 

1 . Updating the Jacobian with the desired positions provides the best control . 

2. Even when using desired or actual positions to update the Jacobian, image noise has a costly effect on the 
resolved motion rate control of a set of features with a small condition number, such as the feature set 
{0,1,2}. Therefore, for control purposes it is best to choose a set of feature points with a small condition 
number, such as the feature set {0,3,5} . 

3. The more points used to estimate the part’s pose, the better the estimate will be. 

In the experiments, the camera was first moved to the desired position over the gasket and the selection criteria 
were evaluated as shown in Table 5. The criteria weighting factors were set to {w, , w 2 , w 3 , w 4 , w 5 , w 6 , w 7 } = 
{0.1, 0.15, 0.05, 0.1, 0.2, 0.2, 0.2}. The computation time of feature extraction was weighted less than the other cri- 
teria* since there was not a noticeable difference in delay time when extracting one circle over the next. The max- 
imum acceptable pose estimation error, condition, and sensitivity were = 100, c mtx = 100,000, and = 68. 
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After the features were selected, the gasket was placed on a conveyor belt moving at a speed of 2.7 centimeters 
per second in the y and z directions of the world coordinates. Using the resolved motion rate visual feedback and a 
feature-based trajectory generator [12], the PUMA robot tracked the gasket. The feature-based trajectory generator 
had a smoothing effect on image noise by spreading out the change in image features over time. To speed up real- 
time computation and avoid estimation noise effects the desired positions of the feature points were used to compute 
the Jacobian. This works satisfactorily if the feature points are already in the vicinity of the desired positions. 

In Figure 4, the first 37 seconds show the ramp response of the system along the z direction while the last 10 
seconds show the steady-state error when the conveyor belt was stopped. The response in the y direction was a simi- 
lar ramp shaped graph. The responses in the x direction and the orientation angles were small oscillatory signals 
about the desired values. The maximum positional errors in the x, y, and z directions were 18.4, 21.6, and 8.0 millim- 
eters, respectively. The maximum orientation errors in roll, pitch, and yaw were 1.5, 2.6, and 2.2 degrees, respec- 
tively. The oscillation in the x position and the orientation angles can be attributed to image noise and the feedback 
delay time. In order to increase the vision sampling, the verification process used to update the position of the circles 
was very simple. Unfortunately, it was also fairly noisy. During the servoing process, the circles in the image were 
located with an accuracy of + 5 pixels. The delay time between when the image was taken and when the robot actua- 
tion began was approximately 100 milliseconds. 

V. Conclusion 

Methodologies for the automatic selection of image features used to visually control the relative pose between 
the manipulator end-effector and a workpiece were developed and analyzed. A resolved motion rate control scheme 
was used to update the robot’s pose based on the position of three features in the camera’s image. The selection of 
these three features for control was based on a weighted criteria function with both image recognition and control 
components. Because of the real-time nature of the control process, it is important to find reliable image features 
which can be quickly extracted from the image. Of particular importance for resolved motion rate control was the 
condition of the Jacobian matrix relating the differential change in the workpiece’s pose to the corresponding dif- 
ferential change in the image feature points. To minimize the effects of image noise on the control, the condition of 
the Jacobian should be minimized. In the simulations and experiments, we found that estimating the pose of the 
workpiece was a time consuming and noise sensitive process. Because of this, the best tracking results were obtained 
by using the desired feature locations to compute the Jacobian instead of the estimated locations. 
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Table 1. Positions and radii of the circles 
with respect to the gasket’s frame. 


Positions and radii in millimeters | 

No. 

X 

y 

z 

radius 

0 

98 

5 

0 

5 

1 

98 

42 

0 

3 

2 

93 

80 

0j 

3 

3 

46 



3 

4 




5 

5 


6 


3 

6 

28 



3 


Table 2. Intrinsic parameters of the camera. 


Parameter 

Symbol 

Value 

Focal length 

/ 



lx 


Y scale factor 

Y, 

EMM 

X focal center 

X 0 

240.0 pixels 

Y focal center 

yo 



Table 3. Ideal case: RMS error of gasket with respect to the camera. 


Control 

Features 

Experiment 

#1 



Jacobian Update 

Actual 

Desired 

Estimated 

{0,3,5} 

RMS 

Estimation 

Error 

Position (mm) 

0 

0 





0.000032 

RMS 

Control 

Error 



0.3526 

0.2847 


0.1493 

0.1722 

0.1493 

{0,1,2} 

rRMS 

Estimation 

Error 

Position (mm) 

0 



Orientation (deg) 

0 

0 

4.5745 

RMS 

Control 

Error 


0.1987 

0.1798 

0.9727 

Orientation (deg) 

0.0360 

0.0430 

0.2940 


* Jacobian updated only if least squares search error F is small. 


Table 4. Image noise case: RMS error of gasket with respect to the camera. The image noise had a Nor- 
mal distribution with zero mean and a standard deviation of 0.5 pixels. The table shows the mean 
position and orientation RMS error of 10 trials. 


Control 

Features 

Experiment 



#3 

#4 

Jacobian Update 

Actual 

Desired 

Estimate #1 

Estimate #2 

{0,3,5} 

RMS 

Estimation 

Error 

Position (mm) 

0 

0 

2.5663 

2.2569 

Orientation (deg) 

0 

0 

2.7393 

2.1801 

RMS 

Control 

Error 

Position (mm) 

1.2708 

1.1542 

1.5735 

1.4578 

Orientation (deg) 

1.3395 

1.2593 

1.5454 

1.4490 

{0,1,2} 

RMS 

Estimation 

Error 

Position (mm) 

0 

0 

OO 

OO 

Orientation (deg) 


0 

oo 

oo 

RMS 

Control 

Error 

Position (mm) 

oo 

OO 

oo 

oo 

! 

Orientation (deg) 

oo 

OO 

oo 

! 

! OO 

i 


* Estimate #1 implies that four circles were used to estimate the gasket’s pose. 
** Estimate #2 implies that seven circles were used to estimate the gasket’s pose. 
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Table 5. The best combinations of features with weights {0.1, 0.15, 0.05, 0.1, 0.2, 0.2, 0.2). 
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